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In this paper, the methods of recursive function theory are used to 
study the size (or cost or complexity) of machines. A positive result of 
this study shows that to a remarkable degree, the relative size of two 
machines is independent of the particular way in which machine size 
is measured. Another result suggests that in order for programs to be 
of economical size, the programming language must be powerful 
enough to compute arbitrary general recursive functions, rather than 
some restricted subset such as the primitive recursive functions. 
Finally, a kind of speedup theorem is proved which is curiously inde- 
pendent of whether the measure of complexity be the size or the num- 
ber of steps taken by the machines that compute the functions. 


INTRODUCTION 


By machines we mean Turing machines, idealized computer programs, 
or any idealized devices for computing the recursive functions. The 
machines and their size must satisfy the axioms of section 1. Section 3 
introduces a general set of axioms for step-counting. These determine 
what is acceptable as a definition of “a step” in a computation. These 
axioms are all so fantastically weak that any reasonable model of a 
computer and any reasonable definition of size and step satisfies them. 

All our examples refer to a specifie class of machines and a specific 
notion of size and step: The class of machines is Davis’ (1958) 1-tape, 
2-symbol unary radix Turing machines, as defined by sets of quadruples. 
The size of a Turing machine is defined to be the number of quadruples 
that define it. A single step in a computation is defined to be a shift of 
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the tape by 1 square (left or right) or a single print or erasure on a 
square. 

Our theorems, if not our examples, refer to arbitrary devices and any 
measures of size and step-counting that satisfy the axioms. 

1. The natural numbers are N = 0, 1, 2, --- . We let (¢;) with 
i ranging over N denote a recursively enumerable (r.e.) sequence of all 
partial recursive (p.r.) functions of one variable. We let (¢;"), k being 
a fixed integer 2 2, denote an r.e. sequence of all p.r. functions of k 
variables. 

Derrtnition (Rogers, 1958). The sequence (¢;) has an acceptable Godel 
numbering if and only if it satisfies the universal Turing machine theorem 
and the iteration theorem, where 

UNIVERSAL TURING MACHINE THEOREM. There exists a partial recursive 
function f such that fli, x) = 9;(x) for all i and x. 

ITERATION THroreM. There exists a recursive function s such that 
g(a, Y) = psa, (y) for all z, z, y. 

THEOREM (Rogers’ 1958). If (¢i) and (pi) both have acceptable Godel 
numberings, then one of these sequences is just a recurswe permutation of 
the other, i.e., there exists a 1-1 onto recursive function f: N — N such that 
fi) = pi for all i. 

This is a deep theorem with immediate consequences, as we shall see, 
for our study of machine size. 

We assume that each ¢; is computed by some set of instructions M; . 
M: may be a Turing machine, a program for an idealized computer, or a 
set of equations. We say that the sequence (M;) has an acceptable Godel 
numbering if (y:) has one, and in the remainder of this paper we always 
assume that (y;) has one. Roger’s theorem allows us to assume, and so 
we do assume without loss of generality, that any two classes of ma- 
chines (M;) and (T:) (both with acceptable Godel numberings) are so 
ordered that M; and T; compute the same function g; . 

Axioms. A recursive function || mapping N (viewed as the set of in- 
dices) — N (viewed as the set of sizes) is called a measure of the size of 
machines, |i | being called the size of M; , if and only if 

(1) there exist at most a finite number of machines of any given size and 
(2) there exists an effective procedure for deciding, for any y, which 
machines are of size y. 

Suppose we compare the machines in one class (M;) with those in 
another (7';). For example, (M:) might be the class of 1-tape 2-symbol 
Turing machines, while (7';) might be the class of 10-tape 10-symbol 
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Turing machines. The following proposition shows that the size of M; 
is approximately equal to the size of T;. It is an immediate corollary 
of Roger’s theorem: 

PROPOSITION 1. Let ||u and ||r be measures of the size of the machines 
(M;) and (T) respectively. Then there exists a recursive function g such 
that for all i: 

1. |i la Sg |i |r (i.e., M: cannot be much larger than Ti) 
2. g\i\lm = |i lr (ie. T; cannot be much larger than M;). 

Proof. Let g(x) = max {gi(x), go(x)} where gi is defined so that 
lila < gı |i|r and gzis defined so that g.|7|u 2 |7|r: To compute 
g(x), note that {z||7|r = z} is finite and may be determined effec- 
tively. Let g(x) = max { |i|x | 7 satisfies |i |r = z}. Let g(x) = 
max { |i |r |i satisfies | i |x = 2}. Q.E.D. 

If M, is much smaller than M; , then one suspects that T; is smaller 
than T; . This is the content of the following: 

PROPOSITION 2. There exists a recursive h such that for all i, 7: 

1. ile S |j lu = lilr £ Ali |e 
2. hlilw S |j lu = |ilr S li lr 

Proof. First note that the g in proposition 1 can be made monotoni- 

cally increasing by setting 
gls) = 1 + max {g(2), go(x), g(x — 1)}. 
We assume that g is monotonically increasing. Set h(x) = gg(x): 


1. If |¢la S ||, then |ilr S gliļu (by 2 of proposition 
1) < g |j |u (because g is monotonically increasing) < gg |j |r (by 1 
of proposition 1) = h |j |r. 

2. If h|iļlmw S |j{w, then by a similar argument g|i|r S 
gg lilu = h|ilļx S |j] S glj|r, and since g is monotonically in- 
creasing, |i |r S |J |r. Q.E.D. 

2. A Turing machine may compute the constant function calx) = 
n by storing all n digits of the response inside its quadruples. In general 
such a machine will be overly large. For example, if n = 10°, a smaller 
machine may output 10° by multiplying 10 by itself 10 times, rather 
than by remembering all 10” digits. We show that any infinite re. se- 
quence of machines contains some machines whose size can be reduced 
in this way. 
THEOREM 1. 1. Let g by any recursive function with infinite range 
(g enumerates indices of an infinite sequence of machines). 
2. Let f be any recursive function. Then there exist i, 7 € N, both 
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uniform in f, g such that 
3. Pi = Cots) 
4. flil < |g(j) | (i.e. M; is considerably smaller than Myn). 
ExamrLE. If f(x) = 100 z, then Mpp is 100 times as large as M; , 
though both Mın) and M; compute the same function. 
Proof. We give a procedure for determining the two integers 7 and j 
uniformly in f, g. Consider the following set of instructions: 
“With inputs z and y, first compute f | y |. Then compute g(0), 
g(1), «++ until the least 7 is found such that f | y | < | g(7) |. Then 
compute y(x) and give gap(£) as output.” 
These instructions define a partial recursive function ¢, (z, y) whose 
index z is uniform in f, g. The recursion theorem’ then supplies an integer 
4 which is uniform in f, g such that 


Equation 1: g(x) = (x, i) for all z. 


We shall show that this is the desired 7. 

Conditions 1, 2 ensure that we can find a j uniformly in f, g that 
satisfies 4, f | i | < | g(j) |: First determine 7, which we have shown to be 
uniform in f, g. Then compare f|7z{| with | g(0) |, |g(1) |, --- until a 
j is found that satisfies 4. For this j (cf. Eq. 1 and above instructions) 
condition 3 is satisfied, 9:() = 97 (a, îi) = ga (£) for all z. Q.E.D. 

g is an algorithmic function for enumerating an infinite set of ma- 
chines. The constructive nature of this proof enables one to effectively 
replace g by a function g’ which enumerates machines that are no larger, 
and sometimes are considerably smaller than those enumerated by g. 

It has been said that since practically all computable functions are 
primitive recursive, one does not need general recursion for any practical 
purposes. Theorem 1, though, gives practical reasons for favoring gen- 
eral recursion: It implies that there exists a primitive recursive function 
whose smallest derivation (defining equations) in the primitive recur- 
sive format is considerably larger than its smallest derivation in the 
general recursive format. More precisely, suppose primitive and general 
recursion are defined by derivations as in Davis (1958). Take the size 
of a derivation to be the number of letters in it. Then each primitive 
recursive function has at least one smallest primitive recursive deriva- 
tion. The set of all such smallest derivations is r.e. ; let g enumerate them. 
Upon setting f(z) = n-z, the theorem supplies a primitive recursive 


1 The recursion theorem asserts that for every partial recursive function h 
there exists an integer 7 which is uniform in h such that g:(x) = A(z, i) for all z. 


ON THE SIZE OF MACHINES 261 


function whose smallest primitive recursive derivation is n times as 
large as its smallest general recursive derivation. The method of theorem 
2 can be used to show that the primitive recursive derivation and the 
much smaller general recursive derivation take approximately the same 
number of steps to compute the same function. 

3. When a machine is reduced in size, it frequently happens that 
the smaller machine takes more steps than the original larger one to do 
its computations. To prepare the way for a study of this phenomenon, 
we now introduce the axioms of Blum (1967) for step-counting. This 
is done by associating with each partial recursive function gy; another 
partial recursive function ®; called its step-counting function. Intui- 
tively, ®,(a) represents the number of steps taken by M; with input x 
to compute o,(2). 

Axroms. Suppose the sequence of all partial recursive functions (gi) 
has an acceptable Godel numbering. Let (®:) denote any r.e. sequence of 
(some but not necessarily all) partial recursive functions. We say that 
(®;) is a sequence of step-counting functions for (¢:) if and only if 

1. g:(x) is defined = &;(x) is defined. 
2. There exists a recursive function R such that 


nian -fi g O= 


The first axiom asserts that if y:(x) is defined, then the number of 
steps required to compute it is finite, and vice versa, if the number of 
steps required to compute ¢;(7) is finite, then ¢,(”) is defined. We shall 
write “@,(xz) < œ” instead of ‘“@,;(x) is defined,” and “&;(x) = œ” 
instead of “@;(xz) is undefined.” The second axiom asserts that there is 
an algorithm for deciding whether or not M; with input z halts in y steps. 
The predicate R of this axiom is similar to Kleene’s T-predicate. 

We note that axioms 1 and 2 are independent: A sequence of step- 
counting functions that satisfies axiom 1 but not 2 is obtained by setting 
®; = g; for all i. On the other hand, ;(z) = 0 for all 7, x satisfies 2 but 
not 1. 

In the following sections, we shall always assume that (®,) is a se- 
quence of step-counting functions for (¢;). The function R will never be 
mentioned, though it is implicit in every statement of the form ‘‘De- 
termine whether ®,(z) = y”. This is determined by computing R(, x, y). 

4. The following theorem, which extends theorem 1, shows that 
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the increase in number of steps that occurs when a machine is reduced in 
size is negligible for all functions that are sufficiently difficult to compute. 

THEOREM 2, There exists a recursive function h such that if 

1. g ts any recursive function with infinite range 

2. f is any recursive function 
then there exist 1,7 € N, both uniform in f, g, such that 

3. Gi = Pa) 

4. fli] < |g) | 

5. for all but a finite number of integers x, [yg (2) defined = (2) < 
h(x, ®g3)(x))] Cie., M; does not take too many more steps than Moin). 

Remark. The function A(x, y) = e(x + y), c = constant, serves for 
the class of Turing machines defined in the introduction. If these Turing 
machines can have an unbounded number of tape symbols, then one 
may choose c = 1. 

Proof. We define a recursive function r. This r has the property that 
for any J, g satisfying 1, 2 there exist integers ¢, 7, in fact, the ones supplied 
by the proof of theorem 1, and there exists an integer w such that 

(a) paa (x) defined => r(w, £, By (a)) = S(x) for all z. 
Out of r we define h to be the recursive function 


h(x, y) = max {r(w, z, y) |w < 2} 


whence it follows that for all sufficiently large x 
(b) gan(z) defined = h(x, Byn(%)) 2 r(w, £, Bac (@))- 
Together, (a) and (b) prove 5. 
DEFINITION OF r(w, £, y). We choose some fixed effective 1-1 onto 
map N >NX.--- XN. The number w is a code word for a 5-tuple 


5 

(wi, +++, Ws). Set f = pwi, 9 = Lup Determine t in terms of these f, g, 
using the algorithm in the proof of theorem 1. This is possible since 
that algorithm defines an index 2 for any partial recursive functions f, g, 
not necessarily satisfying 1, 2 (though 7 is not necessarily defined unless 
1, 2 are satisfied). Now set r(w, 2, y) = ®,(x) if all the following hold: 

(i) a, |i | = ws (sof |i | is defined) 

(ii) Doto duk) = ws (so g(0), -+ , g(ws) are defined) 

(iii) There exists a j such that 7 < wand f|i| < |g(j) | 

(iv) The least j satisfying (iii) satisfies 8y (z) = y 
Set r(w, x, y) = 0 otherwise. 
If (i)-(iv) are satisfied, then S; and xn have the same domain, and 
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a(x) is defined by (iv). We note that if f, g satisfy 1, 2, then there 
exists a j such that |g(j)| > fli]. Hence there exists a w = 
(wi, «++ , Ws), Wi, We being indices of f, g, which satisfies (i)-(ii). For 
this w, equation (a) is satisfied. Q.E.D. 

One can show that theorem 2 is made false by replacing the “for all 
but a finite number of integers x” in 5 by “for all x”. This is interesting 
because theorem 3, which looks so much like theorem 2, does have “for 
all z” in its version of 5. 

Tuuorem 3. 1. Let g be a recursive function satisfying ggn = g; for all j. 

2, Let f, h be recursive functions. 
Then for every i € N there exists aj € N which is uniform in f, h, 9, 1, 
such that 

3. pi = P) 

4. fli] < | gC) | Cie., M; is considerably smaller than Myin) 

5. For all alex) defined = h(x, ®(2%)) < Ëya(x)] (te., Mi takes 
considerably fewer steps than Mac) 

Proof. Select any integer 7. We seek a j for which 3, 4 and 5 are true. 
This j is gotten as follows: We define a partial recursive function r(n, x, y) 
uniformly in f, h, g, i (Eq. 1). By an application of the recursion theorem 
we obtain from it a recursive function q such that gam (x£) = r(n, 2, g(n)) 
(Eq. 2). This q is uniform in f, h, g, i. Finally, we show how to effec- 
tively select n, so that 7 = g(n) satisfies 3, 4, 5. 


Equation 1: r(n, x, y) =^ if flz| = loty) | (ie, if 4 is not 
satisfied with y = 7) 
g(x) if (i) fli| < | gly) | (4 is sat- 
isfied with y = 7) 
and (ii) (x) is defined 
and (iii) A(z, Bi(x)) < anle) (5 
is satisfied with y = 7) 
undefined otherwise. 


This r is computable. The recursion theorem’ supplies a recursive func- 
tion q which is uniform in f, h, g, 2, such that gqm)(%) = r(n, z, q(n)). 


2 The extended recursion theorem states that for every partial recursive func- 
tion r(z1, *** Zm, Y1, *** , Yn, Z) there exists a recursive function g which is 
uniform in 7 such that ggei,+--.am)(Wy *** Yn) = Ty t, Emy Ys tor Yn 
gli, +++» Bm). 
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Plugging into the definition of r, we see that 


Equation 2: gqn)(4) =n if flit] 2 | ga(n) | 
g(x) if (i) flil < | gatn) | 
and (ii) g(x) is defined 
and (iti) A(%,@;,(%)) < Byqiny(2) 
undefined otherwise 
How to Serecr n. First note that f|7| = | gq(n) | means that there 
exists a machine which computes the constant function gyqm(z) = 
Pam (£) = n, whose size is less than a certain fixed integer f | 7 |. This is 
impossible for all n, so there exists an n such that fli] < | gq(n) |. 
To find this value of n, simply test ¢(0), g(1), g(2), --- until an n 
appears that satisfies f| ¿| < | gq(n) |. For this particular choice of n, 
set j = q(n). So 4 is satisfied. It follows that 


Equation 3: y(x) = (2) if (ii) gz) is defined and 
(iii) Ale, B(x)) < Baala) 
undefined otherwise 


5 is satisfied. Suppose ¢.(x) is defined, but that (to the contrary) 
h(x, (x)) Z Baal). Then r(n, z, j) is undefined. Hence o,(x) # 
gol 2) since by.) (%) S h(x, ;(x)) < æ. This contradicts 2: gan = pj. 


3 is satisfied. If g(x) is undefined, then ¢;(x) is undefined (Eq. 3), so 
gotp(%) = o;(2) = g:(x). If g(x) is defined, then the. proof of 5 shows 
that h(a, (2) ) < (2), so (ii), (iii) in Eq. 3 are satisfied and there- 
fore ga (£) = g:(2). Q.E.D. 

Any algorithmic function g for reducing the size of machines can thus 
be effectively replaced by another function g’ that further reduces the 
size and number of steps taken by infinitely many machines. 

We note that theorem 3 is curiously symmetric with respect to size 
4 and steps 5. 
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